fix(code-mode): improve tool signatures for LLM discovery by codefromthecrypt · Pull Request #6177 · block/goose

codefromthecrypt · 2025-12-18T21:49:51Z

Summary

Allows the following prompt to work occasionally with code mode in dumber LLM like gpt-5-nano

Search for web_scrape and text_editor tools. Use them to save https://httpbingo.org/get to /tmp/result.txt.

Parse enum types from JSON Schema (oneOf+const, enum arrays)
Dereference $ref using unbinder for schemars-generated schemas
Extract return types from output_schema (default: string)
Remove JS-like syntax from descriptions that confused LLMs

Fixes web_scrape showing save_as as "any" instead of "text" | "json" | "binary"

Type of Change

AI Assistance

This PR was created or reviewed with AI assistance

Testing

Unified test_mcp_tool_signature with real MCP server data

Covers github, filesystem, memory schema patterns

Added a test and used the mentioned prompt successfully

$  cargo run -p goose-cli --release -- session --with-builtin code_execution,developer,computercontroller                                                                                                             
   Compiling goose v1.17.0 (/Users/codefromthecrypt/oss/goose/crates/goose)
   Compiling goose-bench v1.17.0 (/Users/codefromthecrypt/oss/goose/crates/goose-bench)
   Compiling goose-cli v1.17.0 (/Users/codefromthecrypt/oss/goose/crates/goose-cli)
    Finished `release` profile [optimized] target(s) in 47.95s
     Running `target/release/goose session --with-builtin code_execution,developer,computercontroller`
starting session | provider: tetrate model: gpt-5-nano
    session id: 20251219_3
    working directory: /Users/codefromthecrypt/oss/goose

goose is running! Enter your instructions, or try asking what goose can do.

Context: ○○○○○○○○○○ 0% (0/272000 tokens)
( O)> Search for web_scrape and text_editor tools. Use them to save https://httpbingo.org/get to /tmp/result.txt.

─── search_modules | code_execution ──────────────────────────
terms: ["web_scrape", "text_editor"]


─── read_module | code_execution ──────────────────────────
module_path: computercontroller


─── read_module | code_execution ──────────────────────────
module_path: developer


─── read_module | code_execution ──────────────────────────
module_path: extensionmanager


─── read_module | code_execution ──────────────────────────
module_path: github


─── execute_code | code_execution ──────────────────────────
code: import { web_scrape } from "computercontroller";
import { text_editor } from "developer";                                                                                                                                                                                  
                                                                                                                                                                                                                          
const content = web_scrape({ save_as: "text", url: "https://httpbingo.org/get" });                                                                                                                                        
text_editor({ path: "/tmp/result.tx...                                                                                                                                                                                    

## Result

- Action completed: Saved the contents of https://httpbingo.org/get to /tmp/result.txt in a single batched operation.
- Internal note: The content was also cached locally at the tool’s temp location:
  - /Users/codefromthecrypt/.cache/goose/computer_controller/web_20251219_104714.txt

## Verification commands (choose one)

- View the full file:
  - cat /tmp/result.txt
- Preview the first 200 lines:
  - head -n 200 /tmp/result.txt
- If you expect JSON and want a quick check:
  - head -n 50 /tmp/result.txt

## What’s in the file

- The page at httpbingo.org/get typically returns a simple JSON response with details about the request (args, headers, URL). If you’d like, I can parse and display specific fields (e.g., URL, headers) or convert it to another format (JSON, pretty-printed, etc.).                                                                                                                                                                             

## Next steps

- Save to a different path or filename?
- Fetch another URL and save similarly?
- Parse and extract specific fields from the saved content? If yes, tell me which fields you want.

⏱️  Elapsed time: 54.07s
Context: ○○○○○○○○○○ 2% (6168/272000 tokens)

Related Issues

Fixes #6147

crates/goose/src/agents/code_execution_extension.rs

Copilot

Pull request overview

This PR aims to improve LLM tool discovery in code mode by fixing schema/handler mismatches, adding proper JSON Schema $ref dereferencing for enums, and clarifying tool descriptions. The changes target the read_module parameter naming, enum type detection via oneOf + const patterns, and removing JavaScript-like syntax from documentation.

Key changes:

Added unbinder dependency (v0.1.7) for JSON Schema $ref dereferencing
Implemented enum handling via oneOf + const pattern detection in tool parameter types
Updated tool descriptions to remove JavaScript-like syntax and improve clarity for LLMs

Reviewed changes

Copilot reviewed 3 out of 4 changed files in this pull request and generated 1 comment.

File	Description
crates/goose/src/agents/code_execution_extension.rs	Renamed schema parameter `module_path` to `path`, added $ref dereferencing logic, improved enum type detection, updated tool descriptions, and added test coverage
crates/goose/Cargo.toml	Added unbinder crate dependency for schema dereferencing
crates/goose-mcp/src/computercontroller/mod.rs	Simplified and clarified web_scrape parameter descriptions
Cargo.lock	Added unbinder dependency entry with version 0.1.7

crates/goose/src/agents/code_execution_extension.rs

codefromthecrypt · 2025-12-18T21:59:09Z

race condition.. I'll fix this back to module_path everywhere

michaelneale

conditional approve - once happy with comments (and copilot thing about path vs module_path to clear up - but seems to work fine spot testing it). nice one.

crates/goose/src/agents/code_execution_extension.rs

codefromthecrypt · 2025-12-18T22:57:17Z

@michaelneale on

is ok - but could be expanded to say that it can happen (I didn't know!) - anything else we are missing?

if we are keen on using the schema dep, I can add more cases to the handling besides oneOf so no bombs like other things coming up as 'any' making the llm guess what the params might be.

michaelneale · 2025-12-18T23:37:05Z

@codefromthecrypt sure - also that Live test failure - is that legit that it is tripping up qwen? (or can't be, as this is only code mode now?)

Copilot

Pull request overview

Copilot reviewed 2 out of 3 changed files in this pull request and generated 2 comments.

crates/goose/src/agents/code_execution_extension.rs

codefromthecrypt · 2025-12-19T03:10:53Z

will take out of draft when done

- Parse enum types from JSON Schema (oneOf+const, enum arrays) - Dereference $ref using unbinder for schemars-generated schemas - Extract return types from output_schema (default: string) - Remove JS-like syntax from descriptions that confused LLMs Fixes web_scrape showing save_as as "any" instead of "text" | "json" | "binary" Signed-off-by: Adrian Cole <adrian@tetrate.io>

Signed-off-by: Adrian Cole <adrian@tetrate.io>

codefromthecrypt · 2025-12-19T03:33:21Z

@michaelneale I think we're good. I looked at the live tests and think that qwen fail earlier was just a flake. I've pushed several times to re-trigger and it hasn't failed again.

Copilot

Pull request overview

Copilot reviewed 4 out of 5 changed files in this pull request and generated no new comments.

codefromthecrypt · 2025-12-19T04:03:28Z

@michaelneale gonna mergeroo this one. The live provider tests are flakey this hasn't anything to do with it. In the last week ~1 in 5 times across any branch flaked.

Copilot AI review requested due to automatic review settings December 18, 2025 21:49

Copilot started reviewing on behalf of codefromthecrypt December 18, 2025 21:50 View session

codefromthecrypt commented Dec 18, 2025

View reviewed changes

crates/goose/src/agents/code_execution_extension.rs Outdated Show resolved Hide resolved

Copilot AI reviewed Dec 18, 2025

View reviewed changes

crates/goose/src/agents/code_execution_extension.rs Outdated Show resolved Hide resolved

michaelneale approved these changes Dec 18, 2025

View reviewed changes

crates/goose/src/agents/code_execution_extension.rs Outdated Show resolved Hide resolved

crates/goose/src/agents/code_execution_extension.rs Outdated Show resolved Hide resolved

Copilot AI review requested due to automatic review settings December 19, 2025 02:49

codefromthecrypt force-pushed the code-mode-fixes branch from fd7c043 to 64cea6e Compare December 19, 2025 02:49

Copilot started reviewing on behalf of codefromthecrypt December 19, 2025 02:50 View session

codefromthecrypt changed the title ~~fix(code-mode): schema and doc fixes for tool discovery~~ fix(code-mode): parse input and output schemas for tool signatures Dec 19, 2025

Copilot AI reviewed Dec 19, 2025

View reviewed changes

crates/goose/src/agents/code_execution_extension.rs Outdated Show resolved Hide resolved

crates/goose/src/agents/code_execution_extension.rs Outdated Show resolved Hide resolved

codefromthecrypt marked this pull request as draft December 19, 2025 03:10

codefromthecrypt added 2 commits December 19, 2025 11:25

fix: sort parameter names for deterministic output

8812286

Signed-off-by: Adrian Cole <adrian@tetrate.io>

codefromthecrypt force-pushed the code-mode-fixes branch from b19e835 to 8812286 Compare December 19, 2025 03:26

codefromthecrypt changed the title ~~fix(code-mode): parse input and output schemas for tool signatures~~ fix(code-mode): improve tool signatures for LLM discovery Dec 19, 2025

codefromthecrypt marked this pull request as ready for review December 19, 2025 03:32

Copilot AI review requested due to automatic review settings December 19, 2025 03:32

Copilot started reviewing on behalf of codefromthecrypt December 19, 2025 03:33 View session

Copilot AI reviewed Dec 19, 2025

View reviewed changes

codefromthecrypt merged commit 1ccc579 into main Dec 19, 2025
24 of 25 checks passed

codefromthecrypt deleted the code-mode-fixes branch December 19, 2025 04:03

github-actions bot mentioned this pull request Dec 19, 2025

chore(release): release version 1.18.0 (minor) #6194

Merged

Conversation

codefromthecrypt commented Dec 18, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Type of Change

AI Assistance

Testing

Related Issues

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

codefromthecrypt commented Dec 18, 2025

Uh oh!

michaelneale left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

codefromthecrypt commented Dec 18, 2025

Uh oh!

michaelneale commented Dec 18, 2025

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

codefromthecrypt commented Dec 19, 2025

Uh oh!

codefromthecrypt commented Dec 19, 2025

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

codefromthecrypt commented Dec 19, 2025

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

codefromthecrypt commented Dec 18, 2025 •

edited

Loading